Skip to content

232 revises resize/zoom - #511

Merged
wyli merged 12 commits into
Project-MONAI:masterfrom
wyli:232-revise-resize
Jun 11, 2020
Merged

wyli merged 12 commits into
Project-MONAI:masterfrom
wyli:232-revise-resize

Conversation

@wyli

@wyli wyli commented Jun 9, 2020 •

Copy link
Copy Markdown
Contributor

subtask of #232; fixes #290

Revises zoom/zoomd, resize/resized, randzoom/randzoomd, png_saver to use torch interpolate instead of skimage.transform

Status

ready

Types of changes

  • Breaking change (fix or new feature that would cause existing functionality to change)
  • New tests added to cover the changes
  • Docstrings/Documentation updated

@wyli
wyli requested a review from Nic-Ma June 9, 2020 00:12
@wyli
wyli force-pushed the 232-revise-resize branch from 6a16f42 to 253fe8a Compare June 9, 2020 00:22
@wyli wyli mentioned this pull request Jun 9, 2020
2 tasks done
@wyli
wyli force-pushed the 232-revise-resize branch from 253fe8a to f9ca482 Compare June 9, 2020 09:51

@hjmjohnson hjmjohnson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good to me. Very nice removal of 1 dependency.

@wyli
wyli force-pushed the 232-revise-resize branch from f9ca482 to 59116d2 Compare June 10, 2020 12:50
@wyli
wyli force-pushed the 232-revise-resize branch from 1d8afdb to 96c9401 Compare June 10, 2020 14:11
@wyli

wyli commented Jun 10, 2020

Copy link
Copy Markdown
Contributor Author

@Nic-Ma this is ready for review, please help run the integration tests locally, thanks!

@hjmjohnson

Copy link
Copy Markdown
Contributor

@wyli, Could I run the integration testing so that this PR may be merged? If so, please provide instructions of what you need.

@wyli

wyli commented Jun 10, 2020

Copy link
Copy Markdown
Contributor Author

@wyli, Could I run the integration testing so that this PR may be merged? If so, please provide instructions of what you need.

thanks for asking @hjmjohnson, would be great to run the master branch-only tests ./runtests.sh --coverage with two gpus (preferably tesla v100 cards as it's used by the CI tests)

@hjmjohnson

Copy link
Copy Markdown
Contributor

@wyli I am getting two failures

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.36.06    Driver Version: 450.36.06    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 8000     On   | 00000000:18:00.0  On |                  Off |
| 33%   37C    P8    16W / 260W |    734MiB / 48598MiB |      4%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Quadro RTX 8000     On   | 00000000:3B:00.0 Off |                  Off |
| 33%   38C    P8    14W / 260W |      1MiB / 48601MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

./runtests.sh --coverage

======================================================================
FAIL: test_training (tests.test_integration_segmentation_3d.IntegrationSegmentation3D)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/johnsonhj/src/monai/tests/test_integration_segmentation_3d.py", line 329, in test_training
    np.testing.assert_allclose(ave, s, rtol=5e-3)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.005, atol=0

Mismatched elements: 1 / 1 (100%)
Max absolute difference: 0.02076361
Max relative difference: 0.14461818
 x: array(0.122812)
 y: array(0.143575)

======================================================================
FAIL: test_training (tests.test_integration_workflows.IntegrationWorkflows)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/johnsonhj/src/monai/tests/test_integration_workflows.py", line 276, in test_training
    np.testing.assert_allclose(infer_metric, 0.9224808603525162, rtol=1e-3)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.001, atol=0

Mismatched elements: 1 / 1 (100%)
Max absolute difference: 0.01481684
Max relative difference: 0.01606195
 x: array(0.937298)
 y: array(0.922481)

----------------------------------------------------------------------
Ran 638 tests in 527.801s

@hjmjohnson

Copy link
Copy Markdown
Contributor

@wyli : FYI: The master branch also fails at git hash 7444466cd3d1782444dc98d13d0e8c3ff7add191

python -m tests.test_integration_workflows.IntegrationWorkflows
FAIL: test_training (__main__.IntegrationWorkflows)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/johnsonhj/src/monai/tests/test_integration_workflows.py", line 276, in test_training
    np.testing.assert_allclose(infer_metric, 0.9224808603525162, rtol=1e-3)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.001, atol=0

Mismatched elements: 1 / 1 (100%)
Max absolute difference: 0.01481684
Max relative difference: 0.01606195
 x: array(0.937298)
 y: array(0.922481)

----------------------------------------------------------------------
Ran 1 test in 45.166s


@wyli wyli mentioned this pull request Jun 10, 2020
4 tasks done
@Nic-Ma

Nic-Ma commented Jun 11, 2020

Copy link
Copy Markdown
Contributor

@wyli : FYI: The master branch also fails at git hash 7444466cd3d1782444dc98d13d0e8c3ff7add191

python -m tests.test_integration_workflows.IntegrationWorkflows
FAIL: test_training (__main__.IntegrationWorkflows)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/johnsonhj/src/monai/tests/test_integration_workflows.py", line 276, in test_training
    np.testing.assert_allclose(infer_metric, 0.9224808603525162, rtol=1e-3)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "/home/johnsonhj/.venv/monai37/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 846, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=0.001, atol=0

Mismatched elements: 1 / 1 (100%)
Max absolute difference: 0.01481684
Max relative difference: 0.01606195
 x: array(0.937298)
 y: array(0.922481)

----------------------------------------------------------------------
Ran 1 test in 45.166s

Hi @hjmjohnson ,

Thanks for your help here.
I am fixing the integration test error in PR #534

@Nic-Ma Nic-Ma left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.
And I am wondering why you don't use the AffineTransform to do resize/zoom? I think it can also update the affine matrix.

Comment thread monai/handlers/segmentation_saver.py Resolved
Comment thread monai/data/png_writer.py Resolved
@Nic-Ma

Nic-Ma commented Jun 11, 2020

Copy link
Copy Markdown
Contributor

BTW, I ran the test_integration_classification_2d test, it passed as expected:

python tests/test_integration_classification_2d.py
2020-06-11 08:54:38 URL:https://uc1dda3a4e4c1763afdb3fb2e06c.dl.dropboxusercontent.com/cd/0/inline2/A5eCNnYGNh3H2m7g9NCUbctXqqPerwD9LehHUTzpnX-25Ry7_RkOo-2ZjEXK2S4Y0wM4S252LN2cOSGzHTYAIpmvM9y_nN--0X4I6uDwexCX3lPZWUBHAUgS45nRepwZd9ljHNfsnWhD2fm8LairrKIOnr7Se-w56HR8uN4AySNEEja0WMn90P7FSUy-SqCksUxG5hXzW7wg2WezYpVKDaIqfWYJ_Tv0OGoDt5jQkyKJamDwTBEh2IixWGdYhz-ITB42MrTF22Ms2mePcx6MiVtjIvWsZhebk3pO0Ch1K_JZPwpyXwhNXB-N1I3rpELdFPRvcVxtMGeRv2wnX-XL4Cb7l9fdF-NZZ_yW9V_FZ5oytw/file [61834679/61834679] -> "/tmp/tmpx94_q0aq/MedNIST.tar.gz" [1]
MONAI version: 0+untagged.263.g5c85522
Python version: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)  [GCC 7.3.0]
Numpy version: 1.17.4
Pytorch version: 1.5.0
Ignite version: 0.3.0
----------
Epoch 1/4
epoch 1 average loss:0.8184
saved new best metric model
current epoch 1 current AUC: 0.9979 current accuracy: 0.9641 best AUC: 0.9979 at epoch 1
----------
Epoch 2/4
epoch 2 average loss:0.1730
saved new best metric model
current epoch 2 current AUC: 0.9996 current accuracy: 0.9854 best AUC: 0.9996 at epoch 2
----------
Epoch 3/4
epoch 3 average loss:0.0779
saved new best metric model
current epoch 3 current AUC: 0.9998 current accuracy: 0.9909 best AUC: 0.9998 at epoch 3
----------
Epoch 4/4
epoch 4 average loss:0.0480
saved new best metric model
current epoch 4 current AUC: 0.9999 current accuracy: 0.9942 best AUC: 0.9999 at epoch 4
train completed, best_metric: 0.9999  at epoch: 4
best metric 0.9999425584560568
MONAI version: 0+untagged.263.g5c85522
Python version: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)  [GCC 7.3.0]
Numpy version: 1.17.4
Pytorch version: 1.5.0
Ignite version: 0.3.0
----------
Epoch 1/4
epoch 1 average loss:0.8184
saved new best metric model
current epoch 1 current AUC: 0.9979 current accuracy: 0.9641 best AUC: 0.9979 at epoch 1
----------
Epoch 2/4
epoch 2 average loss:0.1730
saved new best metric model
current epoch 2 current AUC: 0.9996 current accuracy: 0.9854 best AUC: 0.9996 at epoch 2
----------
Epoch 3/4
epoch 3 average loss:0.0779
saved new best metric model
current epoch 3 current AUC: 0.9998 current accuracy: 0.9909 best AUC: 0.9998 at epoch 3
----------
Epoch 4/4
epoch 4 average loss:0.0480
saved new best metric model
current epoch 4 current AUC: 0.9999 current accuracy: 0.9942 best AUC: 0.9999 at epoch 4
train completed, best_metric: 0.9999  at epoch: 4
best metric 0.9999425584560568
.
----------------------------------------------------------------------
Ran 1 test in 262.239s

OK

@wyli

wyli commented Jun 11, 2020

Copy link
Copy Markdown
Contributor Author

BTW, I ran the test_integration_classification_2d test, it passed as expected:

python tests/test_integration_classification_2d.py
2020-06-11 08:54:38 URL:https://uc1dda3a4e4c1763afdb3fb2e06c.dl.dropboxusercontent.com/cd/0/inline2/A5eCNnYGNh3H2m7g9NCUbctXqqPerwD9LehHUTzpnX-25Ry7_RkOo-2ZjEXK2S4Y0wM4S252LN2cOSGzHTYAIpmvM9y_nN--0X4I6uDwexCX3lPZWUBHAUgS45nRepwZd9ljHNfsnWhD2fm8LairrKIOnr7Se-w56HR8uN4AySNEEja0WMn90P7FSUy-SqCksUxG5hXzW7wg2WezYpVKDaIqfWYJ_Tv0OGoDt5jQkyKJamDwTBEh2IixWGdYhz-ITB42MrTF22Ms2mePcx6MiVtjIvWsZhebk3pO0Ch1K_JZPwpyXwhNXB-N1I3rpELdFPRvcVxtMGeRv2wnX-XL4Cb7l9fdF-NZZ_yW9V_FZ5oytw/file [61834679/61834679] -> "/tmp/tmpx94_q0aq/MedNIST.tar.gz" [1]
MONAI version: 0+untagged.263.g5c85522
Python version: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)  [GCC 7.3.0]
Numpy version: 1.17.4
Pytorch version: 1.5.0
Ignite version: 0.3.0
----------
Epoch 1/4
epoch 1 average loss:0.8184
saved new best metric model
current epoch 1 current AUC: 0.9979 current accuracy: 0.9641 best AUC: 0.9979 at epoch 1
----------
Epoch 2/4
epoch 2 average loss:0.1730
saved new best metric model
current epoch 2 current AUC: 0.9996 current accuracy: 0.9854 best AUC: 0.9996 at epoch 2
----------
Epoch 3/4
epoch 3 average loss:0.0779
saved new best metric model
current epoch 3 current AUC: 0.9998 current accuracy: 0.9909 best AUC: 0.9998 at epoch 3
----------
Epoch 4/4
epoch 4 average loss:0.0480
saved new best metric model
current epoch 4 current AUC: 0.9999 current accuracy: 0.9942 best AUC: 0.9999 at epoch 4
train completed, best_metric: 0.9999  at epoch: 4
best metric 0.9999425584560568
MONAI version: 0+untagged.263.g5c85522
Python version: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31)  [GCC 7.3.0]
Numpy version: 1.17.4
Pytorch version: 1.5.0
Ignite version: 0.3.0
----------
Epoch 1/4
epoch 1 average loss:0.8184
saved new best metric model
current epoch 1 current AUC: 0.9979 current accuracy: 0.9641 best AUC: 0.9979 at epoch 1
----------
Epoch 2/4
epoch 2 average loss:0.1730
saved new best metric model
current epoch 2 current AUC: 0.9996 current accuracy: 0.9854 best AUC: 0.9996 at epoch 2
----------
Epoch 3/4
epoch 3 average loss:0.0779
saved new best metric model
current epoch 3 current AUC: 0.9998 current accuracy: 0.9909 best AUC: 0.9998 at epoch 3
----------
Epoch 4/4
epoch 4 average loss:0.0480
saved new best metric model
current epoch 4 current AUC: 0.9999 current accuracy: 0.9942 best AUC: 0.9999 at epoch 4
train completed, best_metric: 0.9999  at epoch: 4
best metric 0.9999425584560568
.
----------------------------------------------------------------------
Ran 1 test in 262.239s

OK

thank you!

@wyli

wyli commented Jun 11, 2020

Copy link
Copy Markdown
Contributor Author

Looks good to me.
And I am wondering why you don't use the AffineTransform to do resize/zoom? I think it can also update the affine matrix.

yes, it's the same logic but nn.functional.interpolate supports more interpolation mode(, and probably faster)

@wyli
wyli merged commit a0ab35a into Project-MONAI:master Jun 11, 2020
@wyli
wyli deleted the 232-revise-resize branch October 6, 2020 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optional dependency cupy is not tested normally

3 participants